Oscillatory Activity and Multisensory Processing

Why does food lose its taste when your nose is stuffed up? Why do turn signals attract the drivers’ attention more when clicking sounds are simultaneously presented? Why are we better at hearing speech in noisy environments when we see the lip movements of the speaker? These are all questions of interest for researchers in the areas of multisensory information processing. Although the scientific study of multisensory integration on the behavioral level has been pursued since psychology became an experimental discipline, the precise neural mechanisms underlying multisensory integration, particularly in the human brain, are to date not well understood. The application of modern techniques and analysis strategies now allows us to uncover the finer details of how the various sensory inputs that continuously enter our nervous system are merged to become coherent multisensory percepts. In this chapter, we summarize and discuss recent studies that suggest that synchronized oscillatory brain activity may be a crucial mechanism for multisensory processing. Traditionally, multisensory integration processes have been considered to take place automatically in a hierarchical manner by progressive convergence of pathways in regions of the association cortex such as the superior temporal sulcus (STS) (Beauchamp, Argall, Bodurka, Duyn, & Martin, 2004; Calvert, 2001; Noesselt et al., 2007) and in specialized subcortical regions such as the superior colliculus (SC) (Stein & Meredith, 1993). Interestingly, the collicular multisensory response patterns were similar in awake and anesthetized animals, which supported the original view that higher cognitive processes are not a prerequisite for multisensory processes (Wallace, Meredith, & Stein, 1998). The assumption that multisensory integration can occur automatically is also supported by behavioral studies, which showed that multisensory interactions were not affected by the voluntary directing of attention (Bertelson, Vroomen, De Gelder, & Driver, 2000). Recently, numerous authors have suggested that a pure convergence model might not be sufficient to account for all aspects of multisensory processing (Ghazanfar & Schroeder, 2006; Kayser & Logothetis, 2007; Lakatos et al., 2009; Senkowski, Schneider, Foxe, & Engel, 2008). First, multimodal interactions and modulation already occur in primary sensory cortices, a result that is difficult to reconcile with the hierarchical convergence model. Second, a convergence scenario does not appear flexible enough to allow for rapid recombination of multisensory signals into completely novel percepts. Third, a hierarchical convergence model does not explain how low-level information about objects can remain accessible, because the highlevel representation is noncompositional and does not explicitly make reference to elementary features. Therefore, an alternative account for multisensory processing has been emerging, where multisensory integration is achieved by flexible synchronization of oscillatory signals (Kayser & Logothetis, 2009; Senkowski et al., 2008). In this chapter we first outline the relevance of synchronized oscillatory activity for a number of cortical processes such as sensory information processing, attentional selection, and working memory and then provide an overview on recent studies supporting the notion that synchrony in neuronal populations is important for multisensory integration. Finally, we pinpoint open questions and future research directions in the emerging field of multisensory processing and oscillatory activity.